INDIANA READING ASSESSMENT - GRADE 1 

TECHNICAL REPORT 



June 30, 2005 



Center for Innovation in Assessment 
Web site: www.cia.indiana.edu 

Email: indianareading@cia.indiana.edu 

Phone: 1-800-377-1409 




Table of Contents 



Executive Summary 2 

Introduction 4 

ROAR System 5 

Validity of the Indiana Reading Assessment - Grade 1 6 

Reliability of the Indiana Reading Assessment - Grade 1 12 

Appendices 

Standards Coverage Chart Appendix A 

Assessment Samples Appendix B 

Test Comparisons Appendix C 

Teacher Feedback Appendix D 

Demographic Data Appendix E 

Descriptive Statistics and Score Distribution Appendix F 




Executive Summary 



Reliability and Validity Results 

A simplistic explanation of validity is that an assessment evaluates what it claims to 
assess. For the Indiana Reading Assessment - Grade 1, research was conducted to 
evaluate how well it assesses various reading skills of grade one students. Multiple 
analyses were conducted to establish the validity of Indiana Reading Assessment - 
Grade 1. While the results of all the analyses were encouraging, the results derived 
from the concurrent validity study were most impressive. All the correlations were 
significant beyond the .05 level and the percentages of agreements were high. Thus, 
the Indiana Reading Assessment - Grade f is a valid instrument for assessing 
phonemic awareness, phonics, vocabulary, and reading comprehension skills. The 
validity of the Indiana Reading Assessment - Grade 1 at assessing overall reading 
ability is also established by the Cronbach’s alpha of .711. 

Establishing reliability is important in showing that an assessment obtains the same 
results when given under the same or similar circumstances. Three types of analysis 
were performed in order to evaluate whether the Indiana Reading Assessment - Grade 
1 was reliable in terms of parallel form reliability, split-half reliability, and inter-rater 
reliability. In all three analyses, the resulting correlations were adequate, with the split- 
half reliability most robust. 

Strengths of the Indiana Reading Assessment - Grade 1 

The Indiana Reading Assessment - Grade 1 was designed specifically for Indiana 
teachers to use in assessing students’ reading abilities as determined by Indiana’s 
Academic Standards for Reading. The completed research also shows that the Indiana 
Reading Assessment - Grade f is a valuable tool for assessing a student’s phonemic 
awareness, phonics, vocabulary, reading comprehension, and overall reading abilities. 

Research Needs 

To date, no research has been conducted on the Indiana Reading Assessment that 
disaggregates data across demographic groups. Studies of this kind are important in 
determining whether there are any biases in assessments. Conducting this type of 
research is done by simply collecting the demographic information of a sample group 
of students and comparing the disaggregated data to national or state averages. With 
the introduction of the No Child Left Behind legislation, reporting progress among all 
groups is necessary. 

With the increased focus on scientifically-based reading research, experimental 
research is being used more to establish the effectiveness of assessments, curricula, 
and programs. While this Technical Report contains results establishing the Indiana 
Reading Assessment - Grade 1 as an effective assessment, the goal of the 
assessment goes beyond simply assessing. The goal is that teachers will identify 
students at-risk and in need of intervention, and provide intervention to enable students 
to realize reading success. Further experimental research is necessary to determine 
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whether educators are effectively using the Indiana Reading Assessment - Grade 1 to 
identify at-risk students and provide intervention. There are a variety of ways this 
research could be conducted. By using the data provided for the predictive validity 
study and comparing similar data to that of a control group, important information can 
be gathered about the effectiveness of the assessment in prompting intervention. 

While the Indiana Reading Assessment - Grade 2 has been established as reliable 
and valid in this report, the ROAR system uses a different form of measurement to 
create student and class reports. Analyses are necessary to establish those reports as 
reliable and can be done with data currently collected. 

Conclusion 

The data in the report show the Indiana Reading Assessment - Grade 1 as both 
reliable and valid. Educators can be confident that the assessment can be used to 
effectively measure students’ reading abilities and that the results are indicative of their 
students’ reading skill levels. 
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Technical Report 

Indiana Reading Assessment - Grade 1 



The Indiana Department of Education, working with Indiana University’s Center for 
Innovation in Assessment, developed the Indiana Reading Assessment - Grade 1 to 
serve as a tool for first grade teachers to use in gaining information about the 
developing reading skills of each of their students. The assessment is administered 
and scored by teachers three times during the academic year (i.e., October, January, 
and April). 

To maximize instructional time and minimize testing time, nearly all portions of the 
assessment are designed for full-class administration. By having teachers score the 
assessment, it is possible for teachers to immediately determine students’ developing 
reading strengths and areas of weakness. Training in how to administer, score, and 
interpret student results is provided through a web site, a CD-ROM presentation, and 
optional face-to-face professional development sessions. 

Broad Coverage of Indiana Academic Standards for Reading and Writing: 

Tasks and items of the Indiana Reading Assessment - Grade 1 and associated 
checklists and supplemental resource activities are designed to provide a broad view 
of student literacy progress that reflects Indiana’s Academic Standards for reading and 
writing at grade one. This coverage is broader than that of commercially available tests 
and therefore more useful to Indiana classroom teachers. The clear link to Indiana’s 
Academic Standards allows easy access to curriculum support materials also matched 
to those standards. 

An overview of this broad coverage can be seen in Appendix A: Standards Coverage Charts. 



Match to Reading Skiiis Categories of the National Reading Panel Study: 

The National Institute of Child Health and Human Development, the National Institute 
for Literacy, and the U. S. Department of Education convened the National Reading 
Panel (NRP) in 1997. Panel members were drawn from several disciplines including 
reading research, medicine, psychology, economics, and classroom teaching. The 
NRP was “charged with reviewing research in reading instruction and identifying 
methods that consistently relate to reading success.” The NRP identified and 
summarized research relating to the acquisition of beginning reading skills under the 
five headings: Phonemic Awareness, Phonics, Fluency, Vocabulary, and Text 
Comprehension. The reading skills organization of the National Reading Panel report 
has become an accepted way to describe and report reading skills. Therefore, this 
report interprets reliability and validity based on this set of reading skills. 

Indiana Reading Assessment - Grade 1 

Three parallel forms of the Indiana Reading Assessment - Grade 1 have been 
developed. There are three administration periods: October, January, and April. Each 
form reflects the reading standards listed in Indiana’s Academic Standards for grade 
one. Test items also reflect the National Reading Panel headings as detailed in the 
above paragraph. Most items are worth one point each; a few open-ended items are 
scored on a rubric as indicated in the table on page 5. 
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Table 1: Indiana Reading Assessment - Grade 1: 
Skill Coverage 



Phonemic Awareness (25 items; 25 points) 


Rhyming Words 


5 items 


5 points 


Beginning Sounds 


5 items 


5 points 


Ending Sounds 


5 items 


5 points 


Phoneme Deletion 


5 items 


5 points 


Phoneme Blending 


5 items 


5 points 


Phonics (16 items; 21 points) 


Letter Recognition 


5 items 


5 points 


Initial Sound Recognition 


5 items 


5 points 


Final Sound Recognition 


5 items 


5 points 


Applied Phonics (using phonics in writing; open 
ended; scored on a rubric) 


1 item 


6 points 


Vocabulary (15 items; 15 points) 


Word Comprehension 


8 items 


8 points 


Sentence Comprehension 


4 items 


5 points 


Text Comprehension (19 items; 20 points) 


Short Passage Comprehension 


15 items 


15 points 


Story Comprehension (mix of open-ended and 
multiple choice items) 


4 items 


5 points 



ROAR System for Generating Class and Student Reports: 

An online score entry tool for generating student and class progress reports is 
available through the Reading Online Assessment Reports (ROAR) System. Teachers 
enter student scores that are used to immediately generate the following reports: 



Class Reports 

Administration Sequence Report: This report 
indicates which students should take each part 
of the Assessment next based on their scores 
from previous parts. 

Ciass Skiiis Reports: These reports show 
class strengths and weaknesses in a variety of 
skills including phonemic awareness, 
comprehension, vocabulary, and phonics. 

These reports will assist teachers as they plan 
instruction that meets the literacy needs of their 
students. 

Ciass Progress Report: This report displays 
the reading skills progress that the class has 
achieved over the course of the school year. It 
compares the scores from the First 
Administration to the current Administration. 



Student Reports 

student Score Report: This report lists a student’s 
scores for all parts of the Assessment that he or 
she has taken and indicates the next part of the 
Assessment that the student should take. 

Student Skiiis Reports: These reports list the 
student’s strengths and weaknesses in a variety of 
skills including phonemic awareness, 
comprehension, vocabulary, and phonics. The 
reports assist teachers as they plan instruction in 
order to meet the literacy needs of their students. 

Student Progress Report: This report displays the 
reading skills progress that the student has 
achieved over the course of the school year. 
Progress is measured in the following areas: 
phonemic awareness, applied phonics skills, 
vocabulary, and comprehension. 
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Establishing and Documenting the Validity of the Indiana Grade 1 Assessment 

Several types of validity have been established for the Indiana Reading Assessment - 
Grade 1. These include: 

1) Content Validity (i.e., the test items address the full range of skills appropriate 
for reading at grade one); 

2) Face Validity (i.e., the tests look like the type of reading material that students 
encounter in grade one); 

3) Construct Validity (i.e., item scores for constructs such as overall reading plus 
sub-constructs such as beginning reading skills or reading comprehension 
correlate with each other at an acceptable level); 

4) Predictive Validity {\.e., performance on the Indiana Grade 1 Reading 
Assessment correlates positively with later scores on standardized reading 
assessments such as the third grade ISTEP+ reading assessment; and 

5) Concurrent Validity (i.e., scores on sections of the Indiana Grade 1 Reading 
Assessment provide information consistent with information provided by other 
valid and reliable reading assessments). 

1) Content Validity: Content validity was obtained by matching items on the Indiana 
Reading Assessment - Grade 1 to Indiana’s Academic Standards for reading in grade 
one. Teacher surveys during the field tests gathered teachers’ opinions on whether 
items matched academic standards and indicators and were at an appropriate level of 
difficulty. Changes were made in a few passages, and changes were made in items 
judged to be overly confusing for some students. 

Additional informal measures were developed for aspects of Indiana’s Academic 
Standards not directly addressed on the scored portion of the Indiana Reading 
Assessment - Grade 1. These informal measures were incorporated into a Resource 
Guide and take the form of teacher checklists and rating rubrics designed for use in 
conjunction with student learning activities (e.g., rhyming exercises, monitoring oral 
reading fluency, and recognizing elements of fiction during discussions). 

A second form of content validity is demonstrated through the match of Indiana 
Reading Assessment - Grade 1 items to reporting categories of the National Reading 
Panel (see earlier discussion on page 4). The scored portion of the assessments relate 
to four reporting categories: Phonemic Awareness, Phonics, Vocabulary, and Text 
Comprehension. These reporting areas are further reinforced with the optional teacher 
checklists and rating rubrics that expand the detail with which teachers can monitor 
skills in phonemic awareness, phonics, vocabulary, and reading comprehension. There 
are additional rating scales in the reporting category of Reading Fluency. 
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2) Face Validity: Because very young students (i.e., first graders) have little familiarity 
with testing, the Indiana Reading Assessment - Grade 1 was designed to look as 
much like actual reading and classroom learning activities as possible. For example, 
letter/sound recognition assessments are in a format similar to learning activity pages 
used in many classrooms. Story comprehension is assessed using actual stories in 
forms similar to those students encounter on class bookshelves and in beginning 
commercial reading materials. Reading comprehension questions are incorporated into 
these booklets. 

In addition to multiple-choice items assessing reading comprehension of short 
passages, some of the reading comprehension items parallel classroom learning 
activities by asking students to draw pictures about stories they have just read and 
attempt a written explanation of what they have drawn. This sort of drawing and 
writing/labeling activity approximates learning activities in many elementary classrooms 
and can be scored with a high degree of inter-rater agreement (see reliability 
discussion that follows). 



Samples of the Assessment can be found in Appendix B. 



3) Construct Validity: To demonstrate construct validity of assessments, test items 
that measure particular constructs or aspects of reading (e.g., beginning reading skills 
such as phonemic awareness and phonics) should correlate highly with each other. 
Cronbach’s alpha analysis is used to determine the consistency of item scores. 
Cronbach’s alpha is essentially a function of the number of items and the average 
inter-correlation among the items. The coefficient indicates how well a set of items 
measures a single unidimensional latent construct. In the case of the Indiana Reading 
Assessment - Grade 1, an analysis was performed on the entire assessment to 
determine the consistency of items designed to measure overall reading ability. A 
coefficient of .70 or higher is considered acceptable. The resulting Cronbach’s alpha of 
.711 is above the acceptable level. This indicates that the items of the Indiana Reading 
Assessment - Grade 1 relate to a single construct (i.e., overall reading ability). 

4) Predictive Vaiidity: In Indiana, all third grade students are required to take ISTEP+ 
(Indiana Statewide Testing for Educational Progress Plus) assessments of reading. The 
ISTEP+ reading measure is a standardized reading test matched to Indiana’s second- 
grade academic standards. The test was developed and validated by CTB McGraw-Hill 
and meets all the technical requirements for a standardized reading measure. One 
form of validity for a classroom reading assessment such as the Indiana Reading 
Assessment - Grade 1 is the degree to which scores correlate with later more 
extensive assessments such as the ISTEP+ assessment. 

Interpreting predictive validity correlations for diagnostic tests is not a simple or 
straightforward process. One expects diagnostic tests to correlate positively with later 
assessments, but not to an extremely high degree. This is because classroom 
diagnostic and monitoring tests are designed for the main purpose of helping 
classroom teachers make a difference-especially with students experiencing difficulty. 
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If the test and teachers are successful, many students who were low-level readers in 
grade one will no longer be at the bottom by grade three. On the other hand, it is 
expected that there will be a moderately positive correlation between early and later 
tests because the differentials between top, middle, and lower level students aren’t 
expected to disappear entirely. Zero order correlations and negative correlations would 
be cause for suspicion that the correlated measures were not assessing the same 
construct. 

In 2001, over 30,000 first grade students from across Indiana took the Indiana Reading 
Assessment - Grade 1. Scores from a randomly selected sample of 2,056 of these 
students were correlated with their later third-grade ISTEP+ reading test scores. Data 
analysis performed by comparing the total scores of the two measures resulted in a 
correlation coefficient of r = .525. A correlation coefficient near .50 is generally 
considered acceptable. 

5) Concurrent Validity: In order to document the concurrent validity of the Indiana 
Reading Assessment - Grade 1, student scores on portions of the Indiana Reading 
Assessment - Grade 1 were compared with those same students’ scores on 
appropriate portions and sub-tests of five other reading assessment measures of 
known validity and reliability. The portions and sub-tests were chosen based on their 
similarities in assessing the same skills as the matching Indiana Reading Assessment 
sub-test. When possible, sub-tests requiring similar tasks were chosen. The five 
measures are: 

1 ) Dynamic Indicators of Basic Early Literacy Skills (DIBELS); 

2) Texas Primary Reading Indicators (TPRI); 

3) Woodcock-Johnson III Tests of Reading Achievement (W-JIII); 

4) Stanford Achievement Test (Reading) 9‘^ edition (SAT-9); and 

5) The Iowa Test of Basic Skills (Word Analysis) (ITBS). 

These five measures were among 28 early reading assessment measures analyzed by 
the University of Oregon’s Institute for the Development of Educational Achievement 
( http://idea.uoreaon.edu/assessment/index.htmn . In its final report, the committee 
judged all of the five measures listed above to be technically sound and to provide 
sufficient evidence for making educational decisions about the early reading of 
children. The U.S. Department of Education refers to this study when discussing 
“scientifically” acceptable ways to evaluate children’s reading gains. 

In order to determine the concurrent validity between the Indiana Reading Assessment 
- Grade 1 and the above measures, trained examiners gave to children portions of the 
Indiana Reading Assessment - Grade 1 and portions of one or more other reading 
assessments on the same day. The grade one participants in the concurrent validity 
studies included 715 students from 38 classrooms in 11 Indiana elementary schools 
selected by the Indiana Department of Education to provide a range of schools that 
represented Indiana students both demographically and geographically. The 
classifications from which schools were drawn included rural, small town, large town, 
mid-sized city, and urban area. Elementary school enrollments ranged from 221 
students to 591 students. 
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School percentages for student eligibility for free or reduced price lunch ranged from 
6% to 91%, and minority enrollment percentages ranged from 5% to 66%. 

The Indiana Reading Assessment - Grade 1 is designed as a criterion-referenced 
assessment in which students are expected to master the skills assessed by the end 
of the school year. It also is designed with simpler portions intended to identify 
students who are at-risk and in need of intervention. Because of these two design 
characteristics, it is often the case that very large percentages (i.e., 50% to 90%) of 
students in first grade may “top out” on simpler portions of the assessment (i.e., ceiling 
effects). This presents a problem when typical correlation computation is used to 
determine concurrent validity between tests. The computation of correlations between 
students’ scores on two different measures is typically built on the assumption of 
normal distributions of scores on both measures. When these assumptions are violated 
(i.e., when very high percentages of scores top out or bottom out on tests), reporting 
simple statistical correlations no longer present a completely accurate picture of the 
concurrent validity existing between two assessments. A quantitative index of the 
degree of decision consistency can still be calculated, however, by comparing percent 
agreement (P^) between the Indiana Reading Assessment - Grade 1 and the five 
measures to which it was compared. 

Concurrent validity information for the Indiana Reading Assessment - Grade 1 is, 
therefore, presented in two analyses in order to provide a full view of how well student 
assessment results compare to results students would receive on other validated 
measures. These two presentations are especially important when standard 
correlational analyses are compromised by large percentages of students “topping-out” 
on both the Indiana Reading Assessment - Grade 1 and on the comparison tests. The 
analyses present: 

1) the percentage of agreement between the Indiana Grade 1 Reading 
Assessment and other measures on which students have demonstrated mastery 
(i.e., usually performance at the 80% level or higher) for a particular skill area 
such as phonemic awareness, phonics, vocabulary or reading comprehension; 
and 

2) the degree to which Indiana Reading Assessment - Grade 1 scores correlate 
positively with other valid measures as compared to how well the other 
measures correlate with each other. 
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Concurrent Validity Data Analysis i - 
Identifying Mastery of Beginning Reading Skills: 

It is important for classroom teachers to identify which students have mastered 
beginning reading skills (i.e., letter identification, recognizing letter/sound 
correspondence, and applied phonics) and which still need attention in these areas. 
The level of agreement about skills mastery between the Indiana Reading Assessment 

- Grade 1 and other validated reading assessments is one type of concurrent validity. 
For purposes of this analysis, students were considered to have demonstrated mastery 
of beginning reading skills if they correctly responded to 80% or more of items dealing 
with beginning reading skills. Comparisons between the Indiana Reading Assessment 

- Grade 1 and other assessments are presented in Table 2 below: 



Table 2: Percent Agreement {Pa) of Beginning Reading Skills 
Grade 1, Administration 





Mastery 

Percentage 


Percent 

Agreement {PaY 


Indiana Assessment (lA) 

(Letter ID, letter/sound & phonics) 


86% 


N/A 


TPRI (letter ID + final consonants) 


96% 


83% 


N = 177 


Woodcock-Johnson III (spelling sounds) 


77% 


80% 


N = 170 


DIBELS^ (letter naming fluency) 


69% 


68% 


N = 169 



The Indiana Reading Assessment - Grade 1 demonstrates high levels of agreement 
with other validated measures in its ability to identify students who have reached 
mastery level along with those who are below the mastery level (“non-masters”) of 
beginning reading skills by the mid-point of grade one. Agreement with the DIBELS 
Letter Naming Fluency sub-test is limited by the test’s unusual variable definition of 
mastery (i.e., 40'^ percentile or better based on local norms). This means the DIBELS 
test will always have an artificial upper limit on the percentage of students who may 
demonstrate mastery. For the purpose of this study, the mastery level was set at a 
score of 57 or higher. This score was derived by setting the top DIBELS score for 
grade one students at the average top DIBELS score (71) of all students in the study 
who scored perfectly on the phonics portion of the Indiana Reading Assessment - 
Grade 1. 

^ Pa is defined as the percent of students who were identified as masters on both forms or non-masters on both 
forms (thus, excluding those students who were identified as masters on one form, but as non-masters on the 
other form). It can be calculated by the following formula: 

P _ /Percent consistent! / Percent consistent \ 

” I mastery decisions J I non-mastery decisions J 

^ DIBELS sets no solid mastery point for this measure. It suggests “Students are considered at low risk if they 
perform above the 40*' percentile using local norms.” 
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Concurrent Validity Data Analysis II - 
Concurrent Validity Correlations 

New reading assessments, such as the Indiana Reading Assessment - Grade 1, 
should correlate positively with established, validated reading measures to 
approximately the same level as those established measures correlate with each other. 
Spearman’s Rho^ correlations were calculated to determine relationships among the 
Indiana Reading Assessment - Grade 1 and established reading skill measures used 
in this study (i.e., W-JIII, TPRI, DIBELS, and SAT-9). Correlations among the 
established measures range from r = .341 to r = .412. These correlations are positive 
and statistically significant at or beyond the p < .01 level. When Indiana Reading 
Assessment sub-test scores are correlated with scores from established reading 
assessments, correlation coefficients are comparable to correlations of existing 
validated measures with each other. In several cases, correlations between Indiana 
Reading Assessment - Grade 1 subtests and other validated measures exceed the 
range of how well these validated measures correlate with each other. 

Table 3a: Concurrent Validity Correlations 
Indiana Reading Assessment - Grade 1: Beginning Reading Skills 



Phonemic Awareness 


Indiana Grade 1 & TPRI Combined Total (For both assessments: phoneme 
blending, beginning sounds, ending sounds, rhyming, and phoneme deletion) 




Phonics 


W-JIII (spelling sounds) & TPRI (letter ID + final consonants) 




Indiana Grade 1 (letter ID, letter/sound & applied phonics) & 
TPRI (letter ID + final consonants) 




Indiana Grade 1 (letter ID, letter/sound & applied phonics) & W-JIII 
(spelling sounds 




DIBELS (LNF) & TPRI (letter ID + final consonants) 




Indiana Grade 1 (letter ID, letter/sound & applied phonics) & DIBELS (LNF) 




Vocabulary 


Indiana Grade 1 (word & sentence comprehension) & W-JIII 
(reading fluency + passage comprehension) 





** correlation significant at the .01 level (two-tailed) 



^ Spearman’s Rho, a non-parametric correlation, was used in these analyses due to the large number of tie-scores 
on the upper ends of the assessments. 
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Table 3b: Concurrent Validity Correlations 
Indiana Reading Assessment - Grade 1: Reading Comprehension Skills 



Reading Comprehension 


TPRI Reading Comprehension & SAT-9 Reading Comprehension 


r= .412** 


Indiana Grade 1 Text & Story Comprehension & TPRI Reading Comprehension 




Indiana Grade 1 Text & Story Comprehension & SAT-9 Reading Comprehension 





** correlation significant at the .01 level (two-tailed) 



Samples of the comparison assessments can be found in Appendix C. 

Establishing and Documenting Reliability of the Indiana Reading Assessment - 
Grade 1 : 

Parallel Form Reliability 

The Indiana Reading Assessment - Grade 1 was designed with parallel forms of the 
assessment for each administration in order to gather information on the progress of 
students’ reading skills. Because of this design, parallel form reliability could be 
established using the equivalent forms of each administration. Students were given 
one administration of the assessment and then administered the next form within a 
limited time span to gather the data necessary to complete the analysis. The Pearson 
analysis produced a reliability coefficient of .68. 

Split-half Reliability 

The split-half reliability coefficients for the Indiana Reading Assessment - Grade 1 
were calculated following the Spearman-Brown split-half reliability procedure. This 
procedure is designed to measure the potential for measurement error due to fatigue, 
level of anxiety, and ordering effects of the items. A reliability correlation of .91 was 
found for the Indiana Reading Assessment - Grade 1. Values of .75 - 1.0 are 
considered excellent. 

Inter-rater Reliability/Agreement of Teacher Scoring 

Inter-rater agreement levels for the three administrations (i.e., October, January, and 
May 2003) of this assessment were determined by correlating teacher scores for 
2500+ assessments to scores on the same assessments provided by trained scorers 
at the Center for Innovation in Assessment. The table that follows summarizes these 
correlations of agreement. 



Table 4: Correlations for Inter-rater Agreement 





October 


January 


May 


Beginning Reading Skills* 


N/A 


N/A 


N/A 


Story Comprehension 


.746 


.702 


.646 



* Only the open-ended items of the Assessment were subject to re-scoring. It is assumed 
that the teacher scoring of the multiple-choice items is correct. 
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Standards Coverage Chart 

Indiana Reading Assessment - Grade 1 Coverage of Indiana’s Grade 1 English/Language Arts Standards 



Grade 1: Standard 1: READING: Word Recognition, Fluency, and Vocabulary Development 
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Concepts About Print 



1.1.1 


Match oral words to printed words. 


1.1.2 


Identify letters, words, and sentenees. 


1.1.3 


Reeognize that sentenees start with eapital letters and 
end with punetuation, such as periods, question marks, 
and exelamation points. 









Phonemic Awareness 



1.1.4 


Distinguish beginning, middle, and ending sounds in 
single-syllable words (words with only one vowel 
sound). 


1.1.5 


Reeognize different vowel sounds in orally stated 
single-syllable words. 


1.1.6 


Reeognize that vowels’ sounds ean be represented by 
different letters. 


1.1.7 


Create and state a series of rhyming words. 


1.1.8 


Add, delete, or ehange sounds to ehange words. 


1.1.9 


Blend two to four phonemes (sounds) into reeognizable 
words. 






























Decoding and Word Recognition 



1.1.10 


Generate the sounds from all the letters and from a 
variety of letter patterns, ineluding eonsonant blends 
and long- and short-vowel patterns {a, e, i, o, u), and 
blend those sounds into reeognizable words. 


1.1.11 


Read eommon sight words (words that are often seen 
and heard). 


1.1.12 


Use phonie and eontext elues as self-correetion 
strategies when reading. 


1.1.13 


Read words by using knowledge of vowel digraphs 
(two vowels that make one sound sueh as the ea in eat) 
and knowledge of how vowel sounds change when 
followed by the letter r (sueh as the ea in the word 
ear). 


1.1.14 


Read eommon word patterns {-ite, -ate). 


1.1.15 


Read aloud smoothly and easily in familiar text. 




y y y 
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1.1.16 


Read and understand simple eompound words 
{birthday, anything) and eontraetions {isn ’t, aren % 
can’t, won’t). 


1 






Identify letters, words, and sentenees. Read and 






1.1.17 


understand root words {look) and their infleetional 
forms {looks, looked, looking). 


1 


1.1.18 


Classify eategories of words. 













s/ 




1 









Grade 1: Standard 2: READING: Reading Comprehension 

Structural Features of Informational and Technical Materials 



1.2.1 


Identify the title, author, illustrator, and table of 
eontents of a reading seleetion. 


n 




■■ 








1.2.2 


Identify text that uses sequenee or other logieal order. 




















Comprehension and Analysis of Grade-Level-Appropriate Text 


1.2.3 


Respond to who, what, when, where, why, and how 
questions and discuss the main idea of what is read. 


■■ 






■ 










1.2.4 


Follow one-step written instructions. 




















1.2.5 


Use context (the meaning of the surrounding text) to 
understand word and sentence meanings. 


■■ 






■ 








■ 


1.2.6 


Confirm predictions about what will happen next in a 
text by identifying key words. 


wm 




■■ 






■ 


1.2.7 


Relate prior knowledge to what is read. 


















1^ 



Grade 1: Standard 3: READING: Literary Response and Analysis 



Narrative Analysis of Grade-Level-Appropriate Text 



1.3.1 


Identify and describe the plot, setting, and character(s) 
in a story. Retell a story’s beginning, middle, and end. 


■■■ 




■■ 








1.3.2 


Describe the roles of authors and illustrators. 




















Grade 1: Standard 4: WRITING: Writing Process 


Organization and Focus 


1.4.1 


Discuss ideas and select a focus for group stories or 
other writing. 








■ 


1.4.2 


Use various organizational strategies to plan writing. 


















1 



Evaluation and Revision 



Revise writing for others to read. 







































































































1.5.1 


Write brief narratives (stories) deseribing an 
experienee. 


1.5.2 


Write brief expository (informational) deseriptions of a 
real objeet, person, plaee, or event, using sensory 
details. 


1.5.3 


Write simple rhymes. 


1.5.4 


Use deseriptive words when writing. 


1.5.5 


Write for different purposes and to a speeifie audienee 
or person. 





■ 


s/ 






s/ 




s/ 


s/ 










■ 



Grade 1: Standard 6: WRITING: Written English Language Conventions 

Handwriting 



1 . 6.1 



Print legibly and spaee letters, words, and sentenees 
appropriately. 


■ 


■ 


■ 


■ 


■ 


■ 




■ 






Sentence Structure 



Write in eomplete sentenees. 



















Grammar 



1.6.3 


Identify and eorreetly use singular and plural nouns 
{dog/ dogs). 


1.6.4 


Identify and eorreetly write eontraetions {isn ’t, aren ’t, 
can ’t). 


1.6.5 


Identify and eorreetly write possessive nouns {cat’s 
meow, girls ’ dresses) and possessive pronouns 
{my/mine, his/her, hers). 





■ 














1 



Punctuation 



1.6.6 


Correetly use periods {lam five.), exelamation points 
{Help!), and question marks {How old are you?) at the 
end of sentenees. 


1 


1 


1 


1 


1 


1 








Capitalization 


1.6.7 


Capitalize the first word of a sentenee, names of 
people, and the pronoun I. 


■ 


■ 


■ 


■ 


■ 


■ 








Spelling 


1.6.8 


Spell three- and four-letter words and grade-level- 
appropriate sight words eorreetly. 


■ 


■ 


■ 


■ 


■ 


■ 






■ 








































































STANDARDS COVERAGE CHART (com.) 



Grade 1: Standard 7: LISTENING AND SPEAKING: Listening and Speaking Skills, Strategies, and 
Applications 



i i ■- m f 

aft o 0 ) Sc ^ 

iEraE og q^E 
< 1)0 

So QIO Q.< (OO 



Comprehension 



1.7.1 


Listen attentively. 
















\.12 


Ask questions for clarification and understanding. 














s/ 


1.7.3 


Give, restate, and follow simple two-step directions. 

















Organization and Delivery of Oral Communication 



Stay on the topic when speaking. 



Use descriptive words when speaking about people, 
places, things, and events. 



Speaking Applications 



1.7.6 


Recite poems, rhymes, songs, and stories. 


1.7.7 


Retell stories using basic story grammar and relating 
the sequence of story events by answering who, what, 
when, where, why, and how questions. 


1.7.8 


Relate an important life event or personal experience in 
a simple sequence. 


1.7.9 


Provide descriptions with careful attention to sensory 
detail. 


1.7.10 


Use visual aids such as pictures and objects to present 
oral information. 









s/ s/ s/ 






y y 
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APPENDIX B: ASSESSMENT SAMPLES - PHONEMIC AWARENESS 







































ASSESSMENT SAMPLES - PHONEMIC AWARENESS (cent.) 
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ASSESSMENT SAMPLES - PHONICS (cent.) 





2 point score 1 point score 0 point score 

The skill is developed. The skill is still The skill is not evident 

The stiident applies the developing. The student The student does not apply 

skill most to all of the time, applies the skill the skill, 

inconsistently. 










ASSESSMENT SAMPLES - VOCABULARY 
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ASSESSMENT SAMPLES - COMPREHENSION (cent.) 
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APPENDIX C: TEST COMPARISONS 



Phonemic Awareness: Rhyming Words 


from Indiana fading Assessment — Grade 1 , 

Beginning Reading Skills - First Administration: 

Directions: The teacher says, “I am going to 
say three words that rhyme: cat, sat, mat. These 
three words rhyme. Can you tell me another 
word (or even a make-believe word) that rhymes 
with cat, sat, mat?” 


from TPRJ Kindergarten Inventory, 

Rhyming, Inventory Task 1 : 

Directions: Teacher says, “Listen to these 
words: , , .All of these words 

rhyme. Can you tell me another real word or 
made-up word that rhymes with 
?” 



Phonemic Awareness: Beginning Sounds 


from Indiana fading Assessment — Grade /, 

Beginning Reading Skills - First Administration: 

Directions: The teacher says, “1 am going to 
say three words. One of these words begins with 
the /b/ sound. Listen to the three words and tell 
me which word begins with the /b/ sound: gjrl, 
balloon, cloud. Which of these three words 
begins with the /b/ sound?” 


from TPRJ Kindergarten Inventory, 

Letter to Sound Linking, Inventory Task 7: 

Directions: Teacher asks, “What is the first 
sound in the word ?” 



Phonemic Awareness: Ending Sounds 


from Indiana fading Assessment — Grade /, 

Beginning Reading Skills - First Administration: 

Directions: The teacher says, “1 am going to 
say three words. One of these words ends with 
the /n/ sound. Listen to the three words and tell 
me which word ends with the /n/ sound: duck, 
dog, sun. Which of these three ends with the /n/ 
sound?” 


from TPRI Kindergarten Inventory, 

Detecting Final Sounds, Inventory Task 5: 

Directions: The teacher says, “Say the word 
. Now say (word) again without the 
(For example, the teacher would say, “Say the 
word rain. Now say rain again without the n.”) Do 
not say the letter name; say the sound of the 
letter. 
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TEST COMPARISONS (com.) 



Phonemic Awareness: Phoneme Deletion 


from Indiana fading Assessment — Grade 1 , 

Beginning Reading Skills - First Administration: 

Directions: The teacher says, “I am going to 
say a word. Then I am going to say a sound that 
is part of that word. I want you to say the word 
back to me without that sound. For example, if I 
sav the word band and then ask vou to sav the 
word without the /b/ sound, you would say and. 
And is the word band without the /b/ sound.” 


from TPRJ 1 st Grade Inventory, 

Detecting Initial Sounds, Inventory Task 3: 

Directions: The teacher says, “Say the word 
.” After the child repeats the word, the 
teacher savs. “Now sav (word) aaain without the 

. (For example, the teacher would say, “Say 

the word nice. Now say nice without the n.”) Do 
not say the letter name; say the sound of the 
letter. 



Phonemic Awareness: Blending Phonemes 


from Indiana fading Assessment — Grade 1 , 

Beginning Reading Skills - First Administration: 

Directions: The teacher says, “I am going to 
say parts of a word. After I say the sounds, I 
want you to blend the parts together and tell me 
what the word is. For example, if I say /d/-/o/-/g/, 
what would you say?” 


from TPRI 1 st Grade Inventory, 

Blending Phonemes, Inventory Task 2: 

Directions: The teacher says, “When 1 say 
r-u-g, 1 know the word is rug.” The teacher says, 
“What would the word be if 1 say ?” Say the 

sound for each letter or cluster of letters, not the 
letter names at approximately 14-second 
intervals. 



Phonics: Letter Recognition 

from Indiana fading Assessment — Grade /, from DIBELS First Grade Benchmark Assessments, 

Beginning Reading Skills - First Administration: Letter Naming Fluency - Benchmark I: 

Directions: Say: In this row, find the letter F. Directions: Teacher shows the student a list of 
When you find the letter F, fill in the circle under letters (upper and lower case mixed together), 
it. Teacher says, “Tell me the names of as many 

letters as you can. When I say begin, start here 
(point to the first letter) and go across the page. 
Point to each letter and tell me the name of that 
letter. If you come to a letter you don’t know. I’ll 
tell it to you. Put your finger on the first letter. 
Ready, begin.” 
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TEST COMPARISONS (com.) 



Phonics: Beginning Sound Recognition 


from Indiana fading Assessment — Grade /, 

Beginning Reading Skills - First Administration: 

Directions: Say: Which of the words in this row 
begins with the same sound as the sound at the 
beginning of the word bed? Fill in the circle 
under the word that begins with the sound that 
you hear at the beginning of the word bed. 


from TPRI Kindergarten Inventory, 

Letter to Sound Linking, Inventory Task 7: 

Directions: The teacher shows the student a set 
of letters. The teacher says, “Point to the first 
letter in the word 



Phonics: Ending Sound Recognition 


from Indiana fading Assessment — Grade 1 , 

Beginning Reading Skills - First Administration: 

Directions: Say: Which of the words in this row 
ends with the same sound as the sound at the 
end of the word cave? Fill in the circle under the 
word that ends with the sound that you hear at 
the end of the word cave. 


from TPRI Kindergarten Inventory, 

Letter to Sound Linking, Inventory Task 7: 

Directions: Before administering this task, place 
the following alphabet letters on the right side of 
the magnetic task board: d,m,t,g,p,k. The student 
is given the first two letters of a word. The 
teacher says, “Can you make the word 



Phonics: Story Comprehension 


from Indiana fading Assessment — Grade /, 

Beginning Reading Skills - First Administration: 

Directions: The teacher says: “Draw a picture 
and write about what the girl likes about her pal 
Al.” For the Applied Phonics score, students are 
asked to read what they have written. Teachers 
measure letter-to-sound correspondence. 


from Woodcock-Johnson III, 

Spelling Sounds, Test 20: 

Directions: Say: I am going to ask you to spell 
some words that are not real words — they are 
nonsense words. Nonsense words may sound 
like “bip,” “ost,” or “mib.” Try to spell the 
nonsense word the way you think it would be 
spelled if it were a real word. 



Vocabulary: Word Comprehension 


from Indiana fading Assessment — Grade /, 

Beginning Reading Skills - First Administration: 

Directions: Say: Put your finger on the picture 
of the boat. Which of the words in this row 
names this picture? Fill in the circle under the 
word that names the picture. 


from Woodcock-Johnson III, 

Passage Comprehension, Test 9: 

Directions: The teacher shows the student a 
page that has words and four pictures. The 
teacher says “Put your finger on the picture that 
the words tell about.” 
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TEST COMPARISONS (com.) 



Vocabulary: Sentence Comprehension 


from Indiana fading Assessment — Grade 1 , 

Beginning Reading Skills - First Administration: 

Directions: Say: Look at the picture in each 
row. Then look at the three sentences beside the 
picture. Find the sentence that tells about the 
picture. Fill in the circle next to the sentence that 
tells about each picture. 


from Woodcock-Johnson III, 

Reading Fluency, Test 2: 

Directions: Say: Start here (point to first 
sentence) and read as many sentences as you 
can. Decide if the answer is “yes” or “no.” After 
you get to the bottom (point to the bottom of the 
first column), go to the top (point to the top of the 
second column). Keep working until I tell you to 
stop. Work as fast as you can without making 
mistakes. (Students are timed.) 



Comprehension: Reading Comprehension 


from Indiana fading Assessment — Grade /, 

Beginning Reading Skills - First Administration: 

Directions: Say: There are five questions after 
each story. You will read each story to yourself. 
Then you will read each question and the 
choices after each question. Fill in the circle 
beside the words that answer each question. 


from SAT 9, 

Reading Comprehension: 

Directions: Say: First read each story; then 
read each question and mark the space for the 
answer you think is right. Keep working until you 
come to the bottom of the page. 



Comprehension: Story Comprehension 

from Indiana fading Assessment — Grade 1 , from TPRI 1st Grade Inventory, 

Beginning Reading Skills - First Administration: Reading Comprehension, Inventory Task 2: 

Directions: Say: You are going to read a story Directions: Reading Accuracy: The teacher 

called Mv Pal Al . This is a story about a girl and says, “I am going to ask you to read a story. This 

her special friend Al. The story tells about the is a story about two friends and the sun. After 

things the girl and Al do together. For the you read it, I will ask you a few questions.” 

Comprehension score, students are asked to 
draw a picture and write an answer to the 
question. Verbal responses are also accepted. 
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Appendix D: Teacher Feedback 

The following teacher feedback data is based on survey results from the 2003-2004 school year. 



Table 5. Helpfulness of each part to the instruction of all students. 



Assessment Section 


Most 

Helpful 


Very 

Helpful 


Helpful 


Somewhat 

Helpful 


Not Very 
Helpful 


Least 

Helpful 


Beginning Reading Skills: Part A 


17% 


16% 


12% 


10% 


12% 


12% 


Beginning Reading Skills: Part B 


9% 


23% 


20% 


14% 


10% 


1 .4% 


Beginning Reading Skills: Part C 


10% 


11% 


23% 


19% 


9% 


10% 


Phonemic Awareness 


18% 


9% 


9% 


14% 


9% 


23% 


Story Comprehension: Listening 


4% 


14% 


14% 


9% 


22% 


19% 


Story Comprehension: Reading 


28% 


15% 


11% 


14% 


15% 


13% 



Table 6. Effectiveness of the Indiana Reading Assessment - Grade 1 and materials. 
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TEACHER FEEDBACK (com.) 



Table 7. Clarity of the Indiana Reading Assessment - Grade 1 and materials. 




Table 8. Is the Indiana Reading Assessment - Grade 1 helpful? 




Table 9. Usefulness of the Indiana Reading Assessment - Grade 1 materials. 






















































TEACHER FEEDBACK (com.) 



Table 10. Amount of time to administer each Part. 
(Data from the 2002-2003 teacher survey.) 



Assessment Section 


Under 20 
minutes 


21-30 

minutes 


31-45 

minutes 


46-60 

minutes 


Over 60 
minutes 


Time to administer Part A (Phonics) 


57% 


27% 


8% 


5% 


1% 


Time to administer Part B (Vocabulary) 


34% 


41% 


17% 


6% 


1% 


Time to administer Part C (Comprehension) 


8% 


37% 


39% 


12% 


3% 


Time to administer Phonemic Awareness 


8% 


29% 


30% 


14% 


5% 


Time to administer Story Comprehension: 
Listening 


15% 


50% 


27% 


6% 


1% 


Time to administer Story Comprehension: 
Reading 


39% 


40% 


12% 


2% 


0.4% 
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Appendix E: Demographic Data 

The following demographic data is derived from an informal study of 125 schools who have 
participated in the Indiana Reading Assessments for three consecutive years beginning with the 
2002-2003 school year. 



Table 11. Locale Data 



Locale 


% of 125 schools in locale 
in 2004-2005 


% of Indiana schools in locale 
in 2004-2005 


1=Large City 


6.03% 


9.99% 


2=Mid-size City 


14.66% 


15.79% 


3= Urban Fringe of Large City 


9.48% 


16.36% 


4= Urban Fringe of Midsize City 


3.45% 


6.78% 


5=Large Town 


3.45% 


2.43% 


6=Small Town 


14.66% 


13.26% 


7=Rural, outside MSA 


34.48% 


18.54% 


8= Rural, inside MSA 


12.93% 


16.83% 



Table 12. Achievement Data 



Criteria 


% of 125 schools meeting 
criteria in 2002-2003 


% of Indiana schools meeting 
criteria in 2004-2005 


Above State Average Percent 
Passing English/Language Arts 
on State Test 


44% 


44.80% 


Scores on State Test Increased 
from 2002-2003 Scores 


Not Applicable 


72.80% 


Scores on State Test Increased 
by 5 points or more from 2002- 
2003 


Not Applicable 


42.40% 
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Table 13. Minority/Ethnicity Data 



Minority Data 


125 schools 


Indiana Public Schools 


% Total Enrollment Minority 
Students 


21.77% 


21.30% 


% of schools with more that 
50% minority enrollment in 
2004-2005 


15.20% 


State Data Not Available 


Ethnicity Category 


Average % of ethnicity 
population in the 125 schools 
in 2004-2005 


State Ethnicity Enrollment 
Not Available 


Asian 


0.72% 




Black 


11.12% 




Native American 


0.21% 




Hispanic 


5.68% 




Mixed Race 


3.57% 




White 


77.90% 





Table 14. Income Level Data 
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APPENDIX F: DESCRIPTIVE STATISTICS 



Appendix F: Descriptive Statistics and Score Distribution 

The following descriptive statistics are based on data from the 2003-2004 Random Sample Group. 
Appendix F-1 Descriptive Statistics and Score Distribution by Reading Skill 



Graph 1 . Phonemic Awareness Score Distribution 




Phonemic Awareness 



Graph 2. Phonics Score Distribution 




Phonics 
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Frequency Frequency 



DESCRIPTIVE STATISTICS (com.) 



Appendix F-1 Descriptive Statistics and Score Distribution by Reading Skill (cont.) 



Graph 3. Vocabulary Score Distribution 




Vocabulary 



Graph 4. Comprehension Score Distribution 




Comprehension 
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Frequency Frequency 



DESCRIPTIVE STATISTICS (com.) 



Appendix F-2 Descriptive Statistics and Score Distribution by Assessment Section 
Graph 5. Beginning Reading Skills Score Distribution 




Mean = 38.2558 
Std. Dev. = 6.1403 
Range = 45 
N = 3,905 



Graph 6. Story Comprehension Score Distribution 




Story Comprehension 
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